SYSTEM AND METHOD FOR JOINT SOURCE-CHANNEL ENCODING, WITH SYMBOL 
DECODING AND ERROR CORRECTION 



This Application is a CIP of Application Serial No. 60/192,215. 

5 

TECHNICAL AREA 

The present invention relates to source symbol encoding, 
decoding and error correction capability in the context of noisy 
10 channels in electronic communication systems. More particularly 
the prefered present invention is a system and method for joint 
source-channel encoding and variable length symbol decoding with 
error correction, comprising arithmetic encoder and combination 
sequential and arithmetic encoded symbol, decoder means. 

15 

BACKGROUND 

With the increasing popularity of mobile communications 
there has come renewed interest in joint source-channel coding . 

20 The reason is that shared mobile communications channels are 

restrictive in terms of bandwidth and suffer from such as fading 
and interference etc., thus making some form of error protection 
essential, particularly where variable length codes are used. 
Further, it is well known that standard approaches to error 

25 correction are expensive, in terms of required bandwidth, hence 

there exists a need for systems and methodology which can provide 
efficient source and channel encoding and symbol decoding with, 
error correction. Viable candidates include a joint source- 
channel encoding system and methodology which utilizes 

30 characteristics of a source, or source encoder, to provide error 
protection . 

As background, it is noted that one of the earliest works 
that examined the effect of errors on variable length codes was 
35 that of Maxted and Robinson ' in an article titled "Error Recovery 
for Variable Length Codes' 1 ^ IEEE Trans, on Information Theory, 
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IT-31, p. 794-801, (Nov. 1985). Corrections and additions to 
said work were provided by Monaco and Lawlor in "Error Recovery 
for Variable Length Codes", IEEE Trans, on Information Theory, 
IT-33, p. 454-456, (May 1987). And said work was later extended 
5 by Soualhi et al. in "Simplified Expression for the Expected 
Error Span Recovery for Variable Length Codes", Intl. J. of 
Electronics, 75, p. 811-816, (Nov. 1989), and by Rahman et al. in 
"Effects of a Binary Symetric Channel on the Synchronization 
Recovery of Variable length Codes", Computer J., 32, p. 246-251, 

10 (Jan. 1989); as well as by Takishima et al. in "Error States and 
Synchronization Recovery for Variable Length Codes", IEEE Trans, 
on Communications, 42, p.' 783-792; as well as by Swaszek et al. 
in "More on the Error Recovery for Variable Length Codes", IEEE 
Trans, on Information Theory, IT-41, p. 2064-2071, (Nov. 1995)., 

15 all of which focused mainly on the resynchronization ability of 
Huffman Codes. 

In terms of joint source channel coding where the source and 
source encoder characteristics are used to provide error 

20 protection, one of the earliest works which incorporated variable 
length codes was that of Sayood, Liu and Gibson in 
"Implementation Issues in MAP Joint Source/Channel Coding", Proc. 
22nd Annular Asilomar Conf. on Circuits, Systems, and Computers, 
p. 102-106, IEEE, (Nov. 1988). Assuming a Markov model for the 

25 source encoder output they used packet izat ion to prevent error 
propagation and the residual redundancy at the source encoder 
output to provide error protection. This approach is used by 
Park and Miller who have developed a bit constrained decoder 
specifically for use with variable codes, (see "Decoding 

30 Entropy-Coded Symbols Over Noisy Channels by MAP Sequency 
Estimation for Asynchronous HMMs", Proc. Conference on 
Information Sciences and Systems, IEEE, (March 1999)., Murad and 
Fuja, in "Robust Transmissions of Variable-Length Encoded 
Sources", Proc. IEEE Wireless and Networking Conf. 1999, (Sept. 
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1999); and Sayood, Otu and Demir in "Joint Source/Channel Coding 
for Variable Length Codes", IEEE Transactions on Communications, 
48:787-794, (May 2000), describe designs which make use of the 
redundancy at the source coder output for error correction. 

The problem of low bandwidth hostile channels can also be 
addressed using error resilent source codes which incorporate the 
possibility of errors in the channel and provide mechanisms for 
error concealment. Work in the area includes that of Yang, et 
al. as reported in "Robust Image Compression Based on Self- 
Synchronizing Huffman Code and Inter-Subband Dependency", Proc. 
thirty-second Asilomar Conference on Signals, Systems and 
Computers, p. 986-972 (Nov. 1997), who use the self- 
synchronizing property of suffix rich Huffman codes to limit 
error propagation, and correlation between subbands to provide 
error correction/concealment. 

In addition, there exist a number of concatenated schemes in 
which the source and channel encoders are concatenated in the 
traditional manner with channel resources allocated between them 
based on the characteristics of the channel. If the channel is 
very noisy, more bits are allocated to the channel and fewer to 
source encoding, and the situation is reversed when the channel 
conditions are more favorable. Examples of this approach include 
the work of Regunathan et al. as presented in an article titled 
"Robust Image Compression for Time Varying Channels", Proc. 
Thirty-first Asilomar Conf. on Signals, Systems and Computers, p. 
968-972, (Nov. 1997) and in an article titled "Progressive Image 
Coding for Noisy Channels", by Sherwood et al., IEEE Signal 
Processing Lett., 4 p. 189-191, (July 1997). 

Most of the schemes referenced above use Huffman coding or 
variants thereof as the variable length coding scheme, however, 
with the increasing popularity of arithmetic coding, there has 
developed interest in joint source channel coding schemes which 



use said arithmetic coding. One such approach is described in 
"Arithmetic Coding Algorithm with Embedded Channel Coding", 
ElMasry, Electronics Lett., 33 p. 1687-1688, (Sept. 1997); and 
another is described in "Integrating Error Detection into 
Arithmetic Coding", Boyd et al., IEEE Transactions on 
Communications, 45(1), p. 1-3, (Jan. 1997). The ElMasry 
approach involves generation of parity bits which are embedded 
into arithmetic coding procedure for error correction. The Boyd 
approach showed that by reserving probability space for a symbol 
which is not in the source alphabet the arithmetic code can be 
used for detecting errors. Reserving probability space for a 
symbol that will never be generated means that less space remains 
for the source alphabet and this translates into a higher coding 
rate. Said overhead, however, is small considering the 
capability of error detection enabled, as described by Kozintsev 
et al. in "Image Transmission Using Arithmetic Coding Based on 
Continuous Error Detection", Proc. of Data Compression Conf. p. 
339-348, IEEE Computer Society Press, (1998) regarding two 
scenarios, (eg. Automatic Repeat Request (ARQ) based 
communications and serially concatenated coding schemes with an 
inner error correction code and an outer error detection code), 
which use error detecting capability of the arithmetic code with 
an error detection space. 

With an eye to the present invention a Key-word Search for 
relevant Patents which involve inner and outer coding, trellis 
coding, data compression, error detection, error correction, 
variable length coding, arithmetic coding, and data transmission 
over noisy channels, has provided: 

Patent No. 5,200,962 to Kao et al. is disclosed as it describes 
data compression with error correction. 

Patent No. 5,745,504 to Bang is disclosed as it describes bit 
error resilient variable length code transmission. 



Patent No. 6,009,203 to Liu et al. is disclosed as it describes 
variable length coding. 

Patent No. 5,233,629 to Paik et al. is disclosed as it concerns 
5 trellis coded quadrature amplitude modulation. 

Patent No. 4,862,464 to Betts et al. is disclosed as it describes 
error detection in digital modems using trellis coding. 

10 Patent No. 5,206,864 to McConnell is disclosed as it concerns 
optimization of error correction, inner and outer coding. 

Patent No. 5,841,794 to Inoue et al. is disclosed as it describes 
error correction for digital data. 

15 

Patent No. 5,910,967 to Vanderaar is disclosed as it describes 
concatenated coding in which an inner code is configured to match 
the needs of an outer code. 

20 Patent No. 5,870,405 to Hardwick et al. is disclosed as it 
describes digital communication over noisy channels. 

Patent No. 5,517,511 to Hardwick et al. is disclosed as it 
describes transmission of digital data over noisy channels. 

25 

Patent No. 5,983,382 to Pauls is disclosed as it concerns outer 
coding. 

Patents disclosed because they discuss Arithmetic coding are: 

30 

Patent No. 5,311,177 to Kimura et al.; 
Patent No. 5,587,710 to Choo et al.; 
Patent No. 5,418,863 to Ando; 
Patent No. 4,295,125 to Langdon, Jr ; 
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Patent No. 4,286,256 to Langdon, Jr. et al; 

Patent No. 5,774,081 to Cheng et al.; 

Patent No. 5,715,332 to Nakanishi; 

Patent No. 5,710,826 to Osawa et al.; 

5 Patent No. 5,317,428 to Osawa et al. 

No known reference or combination of references, however, 

discloses use of a joint source-channel encoding, symbol decoding 

and error correction system comprising encoder means, modulation- 

10 transmission means, and combination sequential, and encoded 

symbol, decoding means; wherein errors detected by the encoded 

o 

^ symbol decoding means are corrected by methodlogy involving the 
changing of bistable elements in said sequential decoder means, 
or selection of a series of sequential bits from a plurality of 
15 said serieses of sequential bits which result from changing 

bistable elements in said sequential decoder means, particularly 
where said encoder means is an arithmetic encoder and encoded 
symbol decoding means comprises arithmetic decoder, and encoded 
symbols are of variable length. 

20 



25 



30 
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DISCLOSURE OF THE INVENTION 



The present invention can be characterized as a system and 
method involving a concatenated scheme in which the functional 
5 roles of both: 

outer code error detection; and 
inner code error correction; 

],q are facilitated by a joint source-channel encoder which comprises 
operational reserved symbol probability space. In a general 
preferred embodiment, outer code error detection is performed by 
O an encoded symbol decoder means, and inner code error correction 

lit is achieved utilizing a sequential decoder means which uses 

H= 15 information from both a channel output and the encoded symbol 

decoder means. In the preferred embodiment the symbol encoder 
•j\ means is an arithmetic encoder means and the encoded symbol 

^ decoder means is an arithmetic decoder means with error detection 

p capability that functions by detecting the presence of encoded 

W 20 reserved symbol (s), which are not allowed as input to said 
source-channel encoder means. 

r "~ The present invention system can be described as a variable 

symbol length, joint source-channel encoding, symbol decoding and 
25 error correction system comprising: 



encoder means; 

modulation-transmission means; and 
combination sequential, and encoded symbol, 
30 decoding means; 

wherein the encoder means comprises input means for accepting a 
sequential plurality of "allowed input symbols" and output means 
for outputting an encoded sequence of bits for allowed symbols 
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input thereinto; where the terminology "allowed input symbols" 
indicates symbols which when detected by the decoding means will 
not typically identify that an error has occurred, (ie. they are 
a part of an allowed alphabet of input symbols). Further, the 
5 symbol encoder means is functionally interconnected to said 

modulation-transmission means such that entry of a symbol to said 
encoder means results in said encoder means outputting an encoded 
sequence of bits therefore into said modulation-transmission 
means, and said modulation-transmission means and combination 

10 sequential, and encoded symbol, decoding means are functionally 
interconnected such that an encoded symbol sequence of bits 
entered to said modulation-transmission means enters said 
combination sequential, and encoded symbol, decoding means. 
Importantly, said sequential decoding means comprises a plurality 

15 of bistable elements. Continuing, said encoded symbol decoding 
means comprises means for initiating an error correction routine 
to the end that, upon the encountering the presence of an 
unexpected encoded reserved symbol: 

20 at least one bistable element in said sequential decoding 

means is changed; or 

selection is made of a series of sequential bits, said 
selection being made from a group consisting of a plurality 
25 of such series of sequential bits which result from the 

changing of bistable elements in said sequential decoder 
means; 

wherein the terminology "reserved symbol" is used to identify a 
30 symbol not allowed as an input symbol to said symbol encoding 
means, and which, upon being detected by the encoded symbol 
decoding means will trigger an error correcting routine. 

Said encoder means can optionally further comprise means for 
generating, and in a sequence expected by said encoded symbol 
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decoding means, outputting an encoded sequence of bits for at 
least one reserved symbol before and/or after an encoded 

allowed input symbol/ which reserved symbol is not 
allowed as an input symbol to said encoder means input means. In 
5 use, the present invention then provides that, upon the detecting 
of the absence of an expected encoded reserved symbol, as well as 
upon detecting the presence of an unexpected encoded reserved 
symbol : 

10 at least one bistable element in said sequential decoding 

means is changed; or 

selection is made of a series of sequential bits, said 
selection being made from a group consisting of a plurality 
15 of such series of sequential bits which result from the 

changing of bistable elements in said sequential decoder 
means . 

(Note that the selections just recited correspond to serial and 
20 parallel decoding error correction schemes, respectively). 

Continuing, where the encoder means and encoded symbol 
decoder means are arithmetic, the present invention joint 
source-channel encoding, decoding and error correction system can 
25 be described as comprising: 

arithmetic encoder means; 
modulation-transmission means; and 
decoding means; 

30 

wherein said arithmetic encoder means comprises input means for 
accepting a sequential plurality of allowed input symbols and 
output means for outputting an encoded sequence of bits for 
allowed symbols input thereinto, and wherein said decoding means 
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comprises a functional combination of: 

a sequential decoder means which comprises a 
sequence of bistable elements, each of which can be 
set to represent encoded symbol bits, and 

an arithmetic decoder means. 

Further, said arithmetic encoder means is functionally 
interconnected to said modulation-transmission means and said 
modulation-transmission means is functionally interconnected to 
said decoding means. In use, said arithmetic encoder means is 
caused to receive a sequence of allowed symbols at its input 
means and provides an encoded sequence of bits for at least some 
thereof, said sequence of bits being caused to arrive at said 
decoding means. Continuing, said arithmetic decoder means has 
error detection means such that in use said arithmetic decoder 
means, upon detecting the presence of an unexpected encoded 
reserved symbol, initiates an error correction routine to the end 
that: 

at least one bistable element in said sequential decoding 
means is changed; or 

selection is made of a series of sequential bits, said 
selection being made from a group consisting of a plurality 
of such series of sequential bits which result from the 
changing of bistable elements in said sequential decoder 
means . 

As described with respect to the first recitation of a present 
invention system, said reserved symbol is not allowed as an input 
symbol to said symbol encoding means, and the selections just 
recited correspond to sequential and parallel error correction 
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schemes , respectively . 

Where the encoder means and encoded symbol decoder means are 
arithmetic, the present invention joint source-channel encoding, 
5 decoding and error correction system can be more precisely 
described as comprising: 

arithmetic encoder means; 
modulation-transmission means; and 
10 decoding means; 



wherein said decoding means comprises a functional combination of 
a sequential decoder means which comprises a sequence of bistable 
elements, each of which can be set to represent encoded symbol 

15 bits, and an arithmetic decoder means. Further, said arithmetic 
encoder means comprises input means for accepting a sequential 
plurality of allowed input symbols and output means for 
outputting an encoded sequence of bits for allowed symbols input 
thereinto, and said arithmetic encoder means further has means 

20 for generating and, in a sequence expected by said arithmetic 
decoder means, outputting an encoded sequence of bits for at 
least one reserved symbol before and/or after an encoded 

allowed input symbol, which reserved symbol is not 
allowed as an input symbol to said arithmetic encoder means input 

25 means. Said arithmetic encoder means is functionally 

interconnected to said modulation-transmission means and said 
modulation-transmission means is functionally interconnected to 
said decoding means. In use said arithmetic encoder means 
receives a sequence of allowed symbols at its input means and 

30 provides an encoded sequence of bits for at least some thereof in 
combination with a sequence of bits which represent at least one 
encoded reserved symbol in a pattern expected by said arithmetic 
decoder means, said sequence of bits being caused to arrive at 
said decoding means. Importantly, said arithmetic decoder means 
has error detection means such that in use said arithmetic 
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decoder means, upon detecting: 



the absence of an expected encoded reserved symbol; or 

5 the presence of an unexpected encoded reserved symbol; 

initiates an error correction routine to the end that: 

at least one bistable element in said sequential decoding 
10 means is changed; or 

selection is made of a series of sequential bits, said 
selection being made from a group consisting of a plurality 
of such series of sequential bits . which result from the 
15 changing of bistable elements in said sequential decoder 

means . 



A method of practicing the present invention, assuming the 
presence of an arithmetic encoder and arithmetic encoded symbol 
20 decoding system, can be recited as: 

a. providing a present invention system as recited above; 

b. inputting a plurality of symbols to the input means of said 
25 arithmetic encoder means; 

c. causing said arithmetic encoder means to encode at least 
some of said plurality of symbols and output bits corresponding 
thereto, optionally intermingled with at least one arithmetic 

30 encoder means generated encoded reserved symbol, into said 
modulation-transmission means; 

d. causing said modulation-transmission means to enter said 
at least some of said plurality of encoded symbols along with 
at least one encoded reserved symbol entered into said 
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modulation-transmission means, into said functional combination 
of said sequential decoder means and arithmetic decoder means; 

e. causing said arithmetic decoder means to, if detecting 
5 non-present expected or present unexpected encoded reserved 
symbol ( s ) : 

change at least one bistable element in said sequential 
decoder means; or 

10 

select a series of sequential bits, said selection being made 
from a group consisting of a plurality of such series of 
sequential bits which result from the changing of bistable 
elements in said sequential decoder means. 

15 

It is noted that the arithmetic encoder means and 
decoding means, which comprises a sequential decoder means and 
an arithmetic decoder means, can be any electronic systems which 
perform the indicated function. 

20 

It is felt beneficial to provide insight to a specific error 
correction procedure which can be performed by the present 
invention. Again, a present invention joint source-channel 
encoding system can be considered to be sequentially comprised 
25 of: 

an arithmetic encoder; 
a modulation-transmission channel; 
a sequential decoder means; and 
an arithmetic decoder; 

30 

said arithmetic encoder feeding to said transmission channel, and 
said transmission channel feeding to said sequential decoder 
means, and said arithmetic decoder having access both from and to 
said sequential decoder means. In use, a sequence of symbols is 
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entered into said arithmetic encoder with the result being that 
said sequence of symbols is encoded and exited therefrom as a 
binary bit stream sequence of +x/es and -x(Es signals, 
corresponding to a string of "1"/ ( "0" ) 1 s and "0"/("l M )'s which 
5 pass through said transmission channel and enter said sequential 
decoder means, (where said x is a fraction). Hard logic circuit 
decisions are made as to the presence of "1"/ ( "0" ) ! s and 
"0"/("l")'s based on said binary bit stream sequence of +x{Es and 
-xvfEs signals while identify decisions based upon signals wherein 

10 "x" is of a value so as to cause the values of +x\[e~s or -x>fEs to 
be within a null zone of + A to - A around 0.0, and identifying 
said decisions as "branch point" decisions in said sequential 
decoder means. Further, monitoring output from said arithmetic 
decoder results in detection of errors, and when an error is 

15 indicated thereby, a "branch point" is identified in said 
sequential decoder means and an inversion of "l"/("0") or 
"0"/("l") based binary bit thereat to "0"/("l") or "l"/("0") is 
effected as an attempt to correct said detected error. 

20 A method of correcting errors in decoded symbols which are 

encoded by an arithmetic encoder in joint source-channel coding 
system, comprises the steps of: 

a. providing a present invention system as recited above; 

25 

b. entering a sequence of symbols into said arithmetic encoder 
such that said sequence of symbols are encoded and exited 
therefrom as a binary bit stream sequence of +x>j"Es and -x\Tes 
signals, corresponding to a string of "l"/("0") f s and "0"/("l")'s 

30 which pass through said transmission channel and enter said 
sequential decoder means, where x is a fraction; 

c. making hard logic circuitry decisions as to the presence of 
"l"/("0")'s and "0"/("l") f s based on said binary bit stream 
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sequence of +x^Es and -xviEs signals while identify decisions 
based upon signals wherein x is of a value so as to cause the 
values of +x\fEs or -xvlss to be within a null zone of + A to - A 
around 0.0, and identifying said decisions as "branch point" 
decisions in said sequential decoder means; 

d. monitoring output from said arithmetic decoder for errors 
and when an error is indicated thereby, identifying a "branch 
point" in said sequential decoder means and correcting the 
"l"/("0") or "0"/("l") based binary bit thereat by inverting it 
to "0"/("l") or "l"/("0") . 

Said method of error correction can involve step d. being 
practiced more than once, with said error correcting method 
further comprising the step of: 

e. defining a tolerable Hamming distance threshold Tc, and 
keeping count of the number Kc of "branch points" in said 
sequential decoder means at which correction of the "l ,f /("0") or 
"0"/("l") based binary bit thereat by inverting to "0"/("l") or 
"l"/("0") has been performed; and 

if Kc exceeds Tc, expanding the null zone by increasing the 
magnitude of A , thereby making available additional "branch 
points" • 

Further, said error correction method can further comprise 
the step of: 

f. determining in a second or greater practice of step e. if 
the identified "branch point" is sequentially prior to the 
"branch point" identified in the immediately previous practice of 
step e. and if so decreasing the value of Kc by 1, otherwise 
increasing the value of Kc by L 
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Alternatively, said error correcting method can involve 
practice of step d. more than once, with said error correcting 
method further comprising the step of: 



5 e. defining a means for calculating a Euclidean distance 
between received and decoded symbols, and a tolerable rate of 
increase of Euclidean distance between sequential practice of 
step d . , and 

10 if said Euclidean distance increases faster than at said 

tolerable rate, expanding the null zone by increasing the 
magnitude of A , thereby making available additional "branch 
points" . 

15 Again, the error detection method, in step d., involves the 

determination of the presence or absence of non-alphabet, (ie. 
reserved), symbols other than as expected, said non-alphabet 
symbols being not-allowed as arithmetic encoder input symbols. 

20 The just decsribed approach to correcting errors requires 

that "branch-points" in the sequential decoder means be 
determined based upon a "null-zone" criteria, and involves 
retracking the contents of the sequential decoder means, and 
selectively changing an identified "l"/("0") to "0"/("l"), when 

25 an error is identifed. It is possible, however, to identify 
bistable elements in said sequential decoder means and define 
them as fixed branch points, based upon the modulation technique 
utilized. For instance, if a trellis coded modulation scheme is 
utilized, a well known 8-PSK Constellation Codeword Assignment 

30 approach can be practiced. When such an approach to correcting 
errors in decoded symbols which are encoded by an arithmetic 
encoder in joint source-channel coding system is utilized, the 
method thereof can be described as comprising the steps of: 

a. providing a joint source-channel encoding, decoding and 
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error correction, system comprising: 



arithmetic encoder means; 
modulation-transmission means; and 
5 decoding means; 

wherein said decoding means comprises a functional combination of 
a sequential decoder means which comprises a sequence of bistable 
elements, each of which can be set to represent encoded symbol 

10 bits, and an arithmetic decoder means. Specific bistable 

elements in said sequential decoder means are identified as 
branch points based upon the modulation scheme utilized, (eg. 
Trellis Coded Modulation and application of 8-PSK Constellation 
Codeword Assignment Criteria). Said arithmetic encoder means 

15 further comprises input means for accepting a sequential 
plurality of allowed input symbols and output means for 
outputting an encoded sequence of bits for allowed symbols input 
thereinto, and said arithmetic encoder means further optionally 
has means for generating and, in a sequence expected by said 

20 arithmetic decoder means, outputting an encoded sequence of bits 
for at least one reserved symbol before and/or after an encoded 

allowed input symbol, which reserved 
symbol is not allowed as an input symbol to said arithmetic 
encoder means input means. Said arithmetic encoder means is 

25 functionally interconnected to said modulation-transmission means 
and said modulation-transmission means is functionally 
interconnected to said decoding means, such that in use, said 
arithmetic encoder means receives a sequence of allowed symbols 
at its input means and provides an encoded sequence of bits for 

30 at least some thereof in optional combination with 

at least one encoded reserved symbol in a 
pattern expected by said arithmetic decoder means, said sequence 
of bits being caused to arrive at said decoding means. Said 
arithmetic decoder means has error detection means such that in 
use said arithmetic decoder means, upon detecting the absence of 
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an expected encoded reserved symbol, or the presence of an 
unexpected encoded reserved symbol, initiates an error correction 
routine to the end that: 

selection is made of a series of sequential bits, said 
selection being made from a group consisting of a plurality 
of such series of sequential bits which result from the 
changing of bistable elements in said sequential decoder 
means at said specific branch points; 

is performed. Said method continues with practice of the 
following steps: 

b. entering a sequence of symbols into said arithmetic encoder 
means such that said sequence of symbols are encoded and exited 
therefrom as a binary bit stream sequence; 

c. monitoring output from said arithmetic decoder means for 
errors; 

d. when an error is detected by said arithmetic decoder means, 
(eg. by detecting a present unexpected or non-present expected 
encoded symbol), by using said specific branch point bistable 
elements in said sequential decoder means, producing a plurality 
of series of sequential bits which result from the changing of 
bistable elements in said sequential decoder means at said 
specific branch points; and 

e. determining which series of sequential bits in said 
produced plurality of series of sequential bits is most likely 
correct. 

Said method of correcting errors in decoded symbols can, in 
step e., involve determining which series of sequential bits in 
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said produced plurality of series of sequential bits is most 
likely correct based on applying at least on selection from the 
group consisting of: 

5 a. eliminating any series of sequential bits which contains an 

encoded reserved symbol; 

b. applying a metric to at least two series of sequential bits 
which do not contain an encoded reserved sysbol, to determine 

10 which of said at least two series of sequential bits is most 
likely correct; 

c. applying an Euclidean metric to at least two series of 
sequential bits which do not contain an encoded reserved sysbol, 

15 to determine which of said at least two series of sequential bits 
is most likely correct. 

Finally, it is specifically noted that, while not limiting, 
it is believed that Patentability is definitely established where 

20 the present invention system is comprised of an arithmetic 

encoder means, in combination with a decoding means which is 
comprised of a functional combination of a sequential decoder 
means and an arithmetic decoder means, wherein in use, error 
correction methodology is initiated upon the detecting, by the 

25 arithmetic decoder means, of a non-expected encoded reserved 
symbol, or the absence of an expected encoded reserved symbol 
sequentially inserted with encoded allowed symbols by the 
arithmetic encoder means. It is also noted that no arithmetic 
encoder means is known which provides operational error detection 

30 space. Computer simulation thereof, and of sequential and 
arithmetic decoder means then serve as example systems. 

The present invention will be better understood by reference 
to the Detailed Description Section in combination with the 
Drawings . 
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SUMMARY OF THE INVENTION 



It is therefore a primary purpose and/or objective of the 
present invention to provide a system comprising an outer symbol 
encoder means which comprises operational error detection space, 
and a combination sequential, and encoded symbol, decoding means, 
wherein said outer encoder means is preferably an arithmetic 
encoder, and the encoded symbol, decoding means is preferably an 
arithmetic decoder. 

It is another purpose of the present invention to disclose 
use of reserved symbols as means to enable encoded symbol, 
decoding means, (arithmetic decoder), to identify errors, said 
identified errors being corrected by the changing of at least one 
bit is an associated sequential decoder means. 

It is another purpose yet of the present invention to teach 
that error detection by an arithmetic decoder means can be based 
on detecting the presence of an unexpected encoded symbol or on 
detecting the absence of an expected encoded symbol. 

It is yet another purpose of the present invention to 
disclose methods of enhancing the operation of the sequential 
decoder means means in correcting of errors involving distance 
calculations, (eg. Hamming and Euclidean distances). 

It is a further purpose of the present invention to identify 
use of "null-zones", or use of modulation technique determined 
specific "branch points" in a sequence of bistable elements in a 
sequential detector means. 

Other purposes and/or objectives will become obvious from a 
reading of the Specification and Claims. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



10 



Fig. la shows a preferred embodiment of the present invention 
system applied in a sequential decoding error correction scheme, 
said system comprising a concatenation of an Outer Arithmetic 
Encoder (1), (which comprises operational Error Detection Space); 
a Modulation-Transmission means (2), (ie. a Channel); and a 
functional combination of an Inner Sequential Decoder (3) and an 
Outer Arithmetic Decoder (4). 

Fig, lb demonstrates a present invention system which is similar 
to that shown in Fig. la, but which is modified to emphasize 
application in practice of a parallel decoding scheme. 

15 Fig. 2 shows a "null zone" in a signal space for assumed binary 
BSPK signalling in an additive white Gaussian noise channel. 

Fig. 3 shows possible signal paths can be represented as a fully 
connected binary Trellis. 

20 

Fig. 4 shows a decoding tree for the Fig. 3 scenario, assuming 
error possibilities at each of the Branch Points. 

Figs. 5 and 6 show additional decoding trees which are described 
in the text. 

Fig. 7 shows and example of null zone expansion for A = 0.1, 

A =0.2, and L = 2 . 

inc nz 

30 Fig. 8a plots the number of symbols decoded vs. the number of 
decode operations, with the horizontal lines represent the 
locations of the errors and the downward spikes indicate when the 
null zone is expanded for a present invention system application. 
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Pig. 8b shows the final magnitude of the null zone denoted by the 
solid "stair step" line, showing the error locations and 
magnitudes . 

5 Fig. 9 shows a plot of the fraction of paths that have not been 
pruned at each point for an application of a present invention 
scheme . 

Figs. 10a, 10b, 10c and lOd show how the decrease in the channel 
10 error probability, and corresponding drop in number of branch 
points, affects the required number of parallel decodings. 

Fig. 11 shows comparison of the present invention Joint Source 
Channel Coding Strategy to three conventional schemes, namely 
15 (4/5) convolutional codes, (one with 16 states and one with 8 
states); and (223,255) Reed Solomon Code. 
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DETAILED DESCRIPTION 



Turning now to the Drawings, there is shown in Fig. la, a 
preferred embodiment of the present invention system which is 
applied in a sequential decoding error correction scheme. Shown 
are a concatenation of an Outer Arithmetic Encoder (1), (which 
comprises operational Error Detection Space); a Modulation- 
Transmission means (2), (ie. a Channel) and a functional 
combination of an Inner Sequential Decoder (3) and an Outer 
Arithmetic Decoder (4). Note said the Sequential (3) and the 
Arithmetic (4) Decoders in combination are herein termed a 
Decoding means. Shown entering to the Arithmetic Encoder (1), 
(at input means for accepting a sequential plurality of allowed 
input symbols), are Symbols (s n ), which Symbols (s^) are members 

of an allowed alphabet. Shown exiting the Arithmetic Encoder 

(1), (at output means for outputting an encoded sequence of bits 

for allowed symbols input to said encoder means), is a binary bit 

stream (x, ) which arrives at the Sequential Decoder (3) from 
k 

Modulation-Transmission means (2) as a sequence of binary bits 

(r ). Exiting said Sequential Decoder (3) is a sequence of 
A 

(x ), with (x ) representing a sequence of hard decisions from 
k k 

the channel output based on optimal symbol by symbol detection. 
A parameter d(r x ) is defined to be a distance, in some sense, 

between the decoded and received signal at a time (k). Where 
convolution decoders are utilized this is often taken to be the 
Hamming Distance, (based on hard decisions (x k ) between received 

and decoded sequences). The various possible decoded sequences 
can be viewed as a binary tree with the correct sequence being a 
path through said tree. 

In use then the Arithmetic Encoder sends a binary stream of 
bits (x ) into the Modulation-Transmission means (2), in some 
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mapped form, (eg. mapped to +/- vis for BPSK signaling). 

(Note: BPSK stands for Binary Phase Shift Keying). 

In the traditional sequential decoding scenario the 
structure of the convolution code imposes a restriction on 
possible decoded sequences and, hence, on possible branch points 
along a path. By discarding branches in which an error has been 
detected, the decoding tree can be pruned such that what is left 
is the decoded sequence with the lowest Hamming Distance from the 
received sequence. The structure of the convolution code then 
defines the valid paths in the tree. The job of the Decoder is 
then primarily to find the valid path that results in a decoded 
sequence with the minimum distance from the received sequence. 

Where Arithmetic Encoders are utilized, the situation is not 
as simple. To apply sequential decoding procedures to the case 
wherein an Arithmetic Encoder is utilized, two considerations 
become important: 

1. need for a condition enabling error detection, (a 
solution thereof comprising a new and novel aspect of the present 
invention ) ; 

and 

2. need for a decoding tree. 

The first requirement is easily satisfied if use is made of 
error detection space in the Arithmetic Encoder, and in fact, it 
is noted that it is satisfied in a stronger manner than where 
convolution encoding is utilized. That is, in the Arithmetic 
Encoding case in which use is made of error detection space, the 
appearance of a symbol corresponding to the error detection space 
is a definite indication of error. 
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The second requirement is not as easily satisfied. This is 
because unlike in the convolution encoder case, the output of the 
arithmetic encoder is not restricted in terms of bit patterns 
which it can output, hence, an associated tree would have each 
5 bit as a branch point and the tree grows exponentially with the 
number of bits in a sequence. Thus it becomes necessary to 
identify specific branch points which are most-likely to be the 
location of error, and to arrive at a more rational code tree. 
Present invention methodology makes use of information available 
10 at the output of the Modulation-Transmission means (2), (ie. a 

Channel), to obtain what are the most likely error branch points. 

Assuming binary BSPK signalling and an additive white 

Gaussian noise channel, the signal space can be represented as 

15 shown in Fig. 2. Further, "hard" decision receiver 

implementation decodes received signals as a "0" or "1" depending 

on whether (r ) is positive or negative. Using the language of 
k 

Gitlin and Ho in "Null Zone Decision Feedback Equalizer 
Incorporating Maximum Likelihood Bit Detection:, IEEE Trans, on 

20 Communications, 23 p. 1243-1250, (Nov. 1975), a region of width 
2 A. around the origin is termed the "null zone". Received 
signals, the signal space projections of which land in said "null 
zone", are, under present invention procedure, identified as 
branch point candidates, and it is noted that the number of so 

25 identified branch points can be increased or decreased by 
adjusting the magnitude of A . 

The number of possible paths can be represented as a fully 
connected binary Trellis, such as shown in 3. The heavy lines in 
30 the Fig. 3 Trellis indicate the path of a decoded signal, and the 
circled nodes indicate where the received signals fell into the 
null zone. The circled nodes, it is specifically noted, are the 
branch points, as defined in present invention methodology. 
Fig. 4 shows the Fig. 3 branch points in a decoding Tree 
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representation. Further, it should be understood that present 
invention methodology requires that the data represented by the 
heavy lines in Fig. 3 be stored in the Sequential Decoder (3) 
shown in Fig. la. 

5 

To aide with understanding, suppose that at point "X" in Fig. 
3 an error is detected. The present invention methodology then 
requires that data stored in the Sequential Decoder (3) be 
modified to attempt to remove the detected error. Present 
10 invention methodology would back up in the Sequential Decoder (3) 
to the Bit identified as " Y " , which Bit "Y" is also indicated in 
Fig. 4, proceeding in Fig. 3 the next Bit is a Branch point 
(BP1), which is again identified in Fig. 4. Note that two 
H possibilities exist at (BP1), and are represented in Fig. 4 as 

^15 (Oil) and (111). Similarly at (BP2) Fig. 4 shows that for each 

Hi 

| s l of the two branches stemming from (Oil) and (111), two Branch 

5] Point possibilities exist, leading to the four possibilities 

it (100) (000), (100) and (000) shown in Fig. 4. (Note that if a 

Hi quadrature scheme were utilized, the Tree of Fig. 4 would have 

L 20 four possibilities at each Branch Point (BP1) and ( BP2 ) ) . 

"3 EI? 

- Jf As a specific example, consider that the output of an 

ess 

(l% Arithmetic Encoder is transmitted using a binary signalling 
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scheme with y/!T = 1. Further consider that said output is 

transmitted over a Modulation-Transmission means (2), (ie. a 
qhannel), which corrupts it with additive noise such that the 
output of a signal receiver would provide: 

R k = {-1.06, -1.06, -0.14, 1.56, -1.11, -1.39... 

... 0.09. 0.04. 0.67. -1.55, 1.03, 0.71} 

If A = 0.1 is chosen for the null zone magnitude, and hard 
decoding is performed on the received values while marking bits 



26 



corresponding to signals that fall in the null zone by a "* M , 
then the following results: 



10 



15 



25 



30 



* * 

^A-0 = {0,0,0,1,0,0,1,1,1,0,1,1}; 

and the Tree of Fig. 5 results. Decoding the received bits using 
an Arithmetic Decoder (4) it is found that reading the sixth bit 
the decodes the symbol corresponding to the error detection 
space. The fact that the symbol corresponding to the error 
detection space has been decoded is denoted in Fig. 5 with an 
"X". If the current path is discarded then the decoding has to 
be terminated. Therefore the path is retraced to the root of 
the Tree and the A is increased so that A = 0.2. This causes 
one of the signals that was originally outside the null zone to 
fall inside the null zone thus providing additional branch points 
so that the decoded bit sequence becomes: 



_ * * 

20 *{\ = o 2 = {0,0,1,1,0,0,1,1,1,0,1,1} 

* 

where denotes the explored branch point and the denotes 

unexplored branch points. 



Fig. 6 shows the new decoding Tree corresponding to said binary 
string. The decoding begins at the root and the error is detected 
as before. However, this time there exists an alternative. 
Instead of progressing along the upper branch, steps are retraced 
to the first branch point and then a path along the lower branch 
is pursued. The new path is denoted in Fig. 6 as a dashed 
line. This time , after the 10 th bit an error is detected. 
Hence, steps are retraced once again to the previous branch point 
and progress is pursued down the dotted line to find the decoded 
sequence . 
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In order to capture an error it is sufficient that A be 
greater than the magnitude of the error. It would seem then 
that selecting a large value for A is desirable, however, as 
already mentioned, such an approach leads to proliferation of 
5 branches in a resulting Tree. Further, it is known that small 

magnitude errors are more likely than are large magnitude errors, 
and as a result large values of A typically do not provide 
significant benefit. Also, it is noted that the probability of 
an error being within the last "n" symbols is: 

10 

l-(l-c) n • 

and as a result it is possible to keep the default value of 
small, and increase it for signals corresponding to the last "n M 
15 symbols when an error is detected. Another consideration is 
that the probablity of an error being in a symbol close to the 
point at which an error is detected is higher than the 
probability of the error being in a symbol further away. With 
that in mind it is again noted that the reason for increasing the 

20 value of A is to increase the number of branch points and that 
if the number of branch points is increased too far, 
computational time can be wasted pursuing wrong paths. This 
leads to the insight that the null zone magnitude can 
beneficially be adjusted in a discriminating manner, and an 

25 algorithm enabling this is: 

let a - be the amount by which A is increased at each step, 
mc 

and 

30 

let L be the number of symbols over which it is proposed to 
nz 

increase the value of A . 

When an error is detected the path thereto is retraced for the 
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purpose of locating a branch point. If sufficient bits 

corresponding to L symbols are retraced then the null zone is 

nz 

expanded by increasing A by /\ ^ for all signals corresponding 

to the bits from the current position to the bits corresponding 

to the farthest symbol forward vhere an error was detected. Next 

the path is proceeded along forward with arithmetic decoding. If 

an error is detected within the next L symbols retracing is 

nz 

pursued through bits corresponding to 2L symbols before 

nz 

expanding the null zone. This time the null zone is incremented 

for all signals from the current position to the farthest 

position where an error was detected by A . Note that this 

^ inc 

has the effect of increasing the null zone within L symbols of 

nz 

the error detection by 2L . If a correct path is still not 

nz 

available on the decoding tree and steps corresponding to 3L 

nz 

have been retraced, the null zone is again incremented to all 
signals from the current location to the farthest point forward 
where an error was detected by ^ i nc * This procedure is 
continued until the correct path is found, or the computational 
burden exceeds a predetermined threshold. In the later case a 
decoding error is declared. By expanding the null zone in this 
manner the result is that null zones at different signal 
locations are approximately proportional to the probability of an 
error being detected, as shown in Fig. 7. Further, if channel 
characteristics are known, the probability of an error escaping 
the null zone can be calculated. For instance, if the channel is 
an AWGN, (ie. Added White Gaussian Noise), channel and the signal 

to noise ratio over the channel is 10 dB then the probability of 

-.4 

a hard decision error is around 10 . The probability of the 

error falling outside of a null zone with A = 0.4 JUT is about 

s 

-14 

10 . If an acceptable packet loss rate for the scheme is 

around 10 6 then an increase in the null zone beyond 0.4 Je~ 

" s 

could not be justified. Because of this condition a maximum 
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null zone, denoted as A (p)# is chosen for a given 

max 

probability of transmission error. 



It is noted that in an arithmetic decoder an error will 
almost always propagate. However, the use of detection space 
essentially guarantees that any error will eventually be 
detected. The "Depth First" algorithm allows correction of the 
errors by exploring branches of a code tree, but said approach 
can become computationally expensive. It is, however, possible 
to prune a code tree in order to reduce the number of 
computations. Several constraints can be used to accomplish said 
pruning the code tree, and the inventors herein have made use of 
the fact that making incorrect "corrections" causes increased 
deviation from a correct path. Detection of proceeding along an 
incorrect path can be accomplished by, for instance, keeping 
track of Hamming distance, and/or keeping track of a Squared 
distance in the Euclidean sense. 

Regarding the Hamming distance approach, keeping continuous 

track of the number of corrections still extant is key, with said 

count being compared against a threshold (T\ ). The value of ( T ) 

h h 

is the maximum Hamming Distance between a received and decoded 

sequence which it is decided can be tolerated. The reasoning is 

that the probability of more errors is less than the probability 

of fewer errors, and that if an additional correction make the 

number of corrections extant greater than (T), then the null 

h 

zone should be expanded by increasing the value of £ * 
Expanding the null zone increases the number of possible branch 
points and this increases the possibilities for decoding 
sequences at a distance ( T h ) or less from the received sequence. 

Regarding the approach based on Euclidean distance, a 
squared distance between received and decoded symbols is 
monitored. A running sum of the distance between the sequential 
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decoder output (x k ) and the received sequence is computed and 

compared to the distance between the output of the hard decision 
decoder (x^) and the received sequence. At a time "n" this is 

accomplished by comparing the Euclidean distance for the 
sequential decoder means: 



-j 771 

where m is the encoded bit sequence length; with a threshold: 

■ T e = —Y:(r k -x k ) 2 

iV Jfe=l 

where a is an experimentally determined offset. The idea is 
that as hard decisions are changed the Euclidean distance between 
decoded and received sequences increases. If the distance 
increases at a high rate it can be detected and is indicative of 
proceeding down a wrong path. If a high rate of increase is 
detected the decoder takes the same action as it did for the case 
where (T^) is exceeded under the Hamming approach. If this 

approach becomes too restrictive, the offset which is considered 
acceptable can be incremented. 

The value of (T ) can be initialized to 1.0 if it is desired 

n 

25 to explore all single error events, with increases in (T ) being 

h 

implemented only when a maximum value of A is applied. It is 
noted, however, that single errors with large A may actually be 
less a problem than double errors with a small Z5u Thus, it can 
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be advantageous to increase the value of ( T h ) before increasing 
A. 

In view of the foregoing, it should be appreciated that 
there are three parameters which can be varied in controlling the 
discard criteria, namely: 
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A ; 

(T ); and 



In the following two present invention application scenarios 
are discussed, namely Breadth First and Depth First. In the 
Depth First approach the complexity depends almost completely on 
the number of symbol decodings that take place during a packet 
decoding. For a Breadth First approach, two major factors affect 
the complexity. The first is the average number of decodings 
that take place during the decoding of a packet, which remains 
less than M times the number of symbols in a given packet. The 
second factor is the sorting that takes place before an 
expansion at a branch point. 

With the foregoing in mind, additional comments are 
appropriate regarding two distinguished approaches to Decoding, 
(ie. Breadth First and Depth First). 



Applying the Breadth First approach, involves fixing the 
size of the null zone prior to decoding. It is desirable to 
keep the null zone small to reduce the number of branch points, 
and hence the amount of computation, small. At the same time it 
is necessary to utilize a null zone sufficiently large that the 
probability of missing an error is below what it is determined 
can be tolerated. Assuming an AWGN channel with a known SNR, & 
can be selected as: 



where m \ is the number of bits per packet, P is the channel 
error probability, and q is the desired lower bound on packet 



BREADTH FIRST CRITERIA 




(1) 



decoding rate. The function Q is given by: 
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For this value of A the average number of branch points can be 
calculated as: 



which simplifies to: 



B(p,g) = m 



(3) 



I where 77 = A(p,g) and m is the average number of 

bits per packet. In this implementation, detection of the error 

j detection symbol by the decoder is used to prune the code tree, 

J and the Euclidean distance between the decoded and received 

£0 sequence is used for selecting the best M paths. However, 

j picking the value of M involves tradeoffs with larger values of M 

f increasing the probability that a correct path will be discarded. 

1 The solution adopted was to first perform decoding using a small 

b va-liie of M. If this does not result in successful decoding then 

25 Mis increased by a value Af- and the procedure is repeated. 

l nc 

Said procedure is repeated "until the packet is decoded or a 

predetermined threshold M is reached, at which point a 

max * 

decoding error is declared. 

It is further noted, in the context of a Breadth First 
approach, that knowing the Modulation Technique applied can allow 
determination of Specifc Bistable Elements in a Sequential 
Decoder means which serve as fixed "Branch Points". Fig. lb 
demonstrates a present invention system applied to practice a 
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method based in the alluded to approach. Fig. lb is much like 

Fig. la, but note that a Modulator (5) is specifically shown 

present between the Arithmetic Encoder (1) and Channel (2), and 

that the Sequential Decoder of Fig. la is identified as further 

5 comprising a Demodulator (Demod). Note also that a binary bit 

stream (x t ) enters Modulator (5), but that a modulated binary bit 
k 

stream (y^ enters the Channel (2) with the sequential signals 

entered to the Demod/Sequent ial Decoder being identified by 

(r ). rather than (x, ). It is not felt necesary to provide 

10 1 k 

additional Figures to aide with understanding of the effects of 

this. Rather, referral to Figs. 3-5 shows that various series of 

sequential bits result where bits are changed at various Bistable 

Element "Branch Points", and said various series of sequential 

bits can be demonstrated as Branches of a Tree. It is possible 

1 5 

to, when an error is determined by detection of the presence of 
an unexpected, or the absence of an expected encoded reserved 
symbol by an Arithnetic Decoder means, to form a plurality of 
possible series(es) of sequential bits, followed by: 

20 

selection being made of a series of sequential bits, said 
selection being made from a group consisting of a plurality 
of such seriesof sequential bits which result from the 
changing of bistable elements in said sequential decoder 
25 means, (ie. the various branches of a Tree). 

Of course the selected series of sequential bits will be 
determined by at least one criteria being met, said criteria 
being for instance: 

30 

a* any series of sequential bits which contains an encoded 
reserved symbol is eliminated; 

b. a metric is applied to at least two series of sequential 
bits which do not contain an encoded reserved sysbol, to the end 
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that determination is made regarding which of said at least two 
series of sequential bits is most likely correct; 



c. an Euclidean metric is applied to at least two series of 
5 sequential bits which do not contain an encoded reserved sysbol, 
to the end that determination is made regarding which of said at 
least two series of sequential bits is most likely correct. 
Where this approach is embodied, a suitable decoder, (ideal for 
use with system which utilizes Trellis Coded Modulation ( TCM ) ) , 

10 is a List Viterbi decoder where the size of the list can be 

adjusted to provide increased error protection, with, of course, 
increased complexity. The Kozintsev et al. article titled 
"Image Transmission Using Arithmetic Coding Based on Continuous 
Error Detection", which was cited in the Background Section, it 

15 is noted, reported utilizing arithmetic coding for additional 
error correction in a primarily serially concatenated inner 
convolut ional code scheme. Kozintsev et al., however, did not 
suggest abandoning punctured convolut ional codes as the basis of 
the bulk of their error correction methodology. 
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DEPTH FIRST CRITERIA 



To implement the Depth First approach the parameters 
25 required are: 

initial value of the null zone A ; 



the value by which the null zone is to be incremented a . ; 

* " inc 



the "backtrack" distance L ; 

nz 



the threshold to be used to prune (T ); and 

n 
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the parameter a to be used in constructing a threshold for 
Euclidean distance. 
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It has been found useful to define two thresholds T and T 

h,t h,w 

for Hamming distance and two thresholds n and for the 

Euclidean distance. The total Hamming distance between the 

decoded sequence and the sequence obtained by hard decision 

decoding to the threshold T as previously described. The 

h, t 

Hamming distance between the decoded sequence on the code Tree 

and the sequence obtained by hard decision decoding in a sliding 

window of size L to the threshold T . The end point of the 

w h,w 

sliding window is the current bit. A similar procedure is used 
for the Euclidean distance. It is noted that the values of T 

h, t 

15 and T L are obtained using two estimates of channel noise 
h, w 

variance, one for the entire received sequence . <j2 , and one for 
the sliding window of size L y . The variance a\ is translated 
into a channel probability error "p", and the two thresholds are 
obtained as: 
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T Kt = np(l + 4o t ), T htW = L wP (l + 8a w ) 



In a specific case, the length of the sliding window L was set 

w 

to 50, and both the T. parameters were set to a minimum default 

h 

value of 2. The T^ parameters were found by hard decision 

decoding to produce X and then setting the at and a w to 0.2 and 

2.0 respectively. The value of A was initially set to 0.10 VeT. 

v s 

When the decoder backtracked a symbol distance of 

L =5, the value of A was increased by /\ . = 0.10/1T to a 
nz inc i s 

maximum of 0.70^eT. If the decoder backtracked to the root of 
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the code Tree, the values of T 



and T 



e,t 




trz* e,w - 



( I +- O • l) 



A/ 



5 



were increased by 10% and the values of T 



and T 



were 



h , w 



e , w 



increased by 20%. 

Computational effort was determined by computing the ratio 
of the total number of decode operations performed by the decoder 
to the number of symbols transmitted. In the case where no 
errors occurred this ratio is one. When an error is detected, 
because of backtracks, the decoding scheme requires more decode 
operations than the number of symbols transmitted resulting in a 
value greater than one. When said ratio exceeded 10 3 a decoding 
failure was declared. 

Figs. 8a and 8b present an example of the progression of the 

decoding procedure. Fig. 8a plots the number of symbols decoded 

vs. the number of decode operations. The horizontal lines 

represent the locations of the errors and the downward spikes 

indicate when the null zone is expanded. The trace progresses 

upward in a linear fashion until an error is detected. At that 

point the decoder backtracks. The is indicated by the downward 

slope of the decode trace. The trace continues downward until a 

branch point is reached. Then the trace slopes upward once again 

until another error is detected or the sequence is terminated. 

Note that there can be a lot of "jostling" around the errors 

until the correct branch point is found. Also note that the null 

zone is only expanded when the progression backtracks for a 

symbol distance of L . Fig. 8b shows the final magnitude of 

nz 

the null zone denoted by the solid "stair step" line. Also shown 
are the error locations and magnitudes. These are the peaks 
along the bottom. To obtain a correct decoding, the null zone 
region must be expanded to encompass all the introduced errors. 
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Table 1 presents the results of using the depth first 
decoding approach in terms of packet recovery rates for the four 
different values of the error detection space: 

( ' e = 0.5, 0.29, 0.16, 0.08) . 



10 



15 





Pc = lO" 1 - 5 


Pc = lO" 20 


p c = lO" 25 


p c = lO" 30 


-NONE- 


0.00 


0.00 


0.01 


24.64 


e = 0.08 


0.00 


0.39 


46.63 


96.72 


e = 0.16 


0.00 


17.04 


95.94 


99.17 


e = 0.29 


0.00 


71.09 


99.21 


99.56 


e = 0.50 


0.19 


88.23 


99.51 


99.66 



Table 1: Packet Recovery Rates for Depth First Decoding. 



For comparison, also included is the case where the standard 
20 arithmetic encoder is used, albeit with packet izat ion . The 

results show a more than 99% recovery rate for £ = 0.16, 0.29, 
and 0.5; at a channel error rate of 10~ 3 Similarly high results 
hold for €/= 0.29 and 0.5. for p c = 10" 2 ' 5 ; where p c is the 
probability of error for a symbol being transmitted over the 
25 channel. 

However, for higher error rates the recovery rates drop 
significantly. Note that for a given channel error probability 
the amount of error space that is used is inversely proportional 

30 

to the probability of packet loss. 

To implement the Breadth First approach various parameter 

values were selected as follows: M = 200, jvf = 1800, and 

i nc 

M = 2000. A was chosen to be {1.20, 1.00, 0.91, 0.82} for 
max 
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channel error probabilities of {10 1 ' 5 , 10 2 , 10 2 * 5 , 10 3 }, 
respectively. The parameters used give the lower bounds on 
packet loss rates of {lO" 1 * 5 , 10~ 3 , 10~ 4 , 10~ 5 }, respectively. 
It should be recalled that he algorithm functions by first 
listing all possible paths at a branch point, then pruning all 
but the M which are closest in Euclidean distance, to the 
received sequence. Between the branch point paths get pruned 
because progressing along them results in the decoding of the 
error of the detection space. 



Fig. 9 shows a plot of the fraction of paths that have not 
been pruned at each point. The average fraction of parallel 
decodings used is also indicated on Fig. 9 by the horizontal 
line. Note that more than half of the M paths are valid when a 
15 branch point is reached, the number of branches after the point 

is greater than M. The algorithm will then prune the paths 
furthest from the received sequence. If the valid path is ever 
pruned, the sequence will not decode at all or, with a small 
probability, it will decode incorrectly. 

20 

Figs. 10a, 10b, 10c and lOd show how the decrease in the 
channel error probability, and corresponding drop in number of 
branch points, affects the required number of parallel decodings. 
Also note that the histogram for e =0.5 shifts quite 

25 significantly, dropping to around 50%. However, the distribution 
for e = 0.08 is concatenated on the left and the reduction on 
branch points has little effect. Using Af = 200 to decode the 
packet results in being unable to decode the packet, hence the 
value of M was increased to 2000. If thereafter the packet could 

30 still not be decoded, a decoding failure was declared. For 

channel error probabilities of 10" 3 and 10~ 2 ' 5 a higher number of 

paths was used for less than one packet in 20,000 for all values 

of the error detection space. For a channel error probability of 
_2 

10 the same result held for e = 0.5 and c = 0.29. For e = 0.16 
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the higher number of paths was used for about 20% of the packets 

and about 7% of the packets could not be decoded even with the 

higher number of paths. The situation was significantly worse 

for 6 = 0.08 with only 20% of the packets decoded using M = 200. 

- 1 5 

5 For a channel error rate of 10 and e .= 0.5, a little more 

that 20% of the packets were decoded using M = 200 while about 
50% of the packets were decoded with M = 2000. For other values 
of almost all the packets that could be recovered required M = 
2000. 

10 

Table 2 presents recovery rates for the case where Breadth 
First decoding was applied. 
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20 





Pc = lO- 15 


Pc = lO" 20 


p c = lO" 25 


p c = lO" 30 


-NONE- 


0.00 


0.00 


0.01 


24.64 


e = 0.08 


0.00 


38.53 


99.89 


99.99 


e = 0.16 


0.00 


92.03 


99.94 


99.99 


e = 0.29 


16.63 


99.30 


99.95 


99.99 


e = 0.50 


73.24 


99.33 


99.95 


99.99 



Table 2: Packet Recovery Rates for Breadth First-Decoding 



25 

The recovers rate is greater than 99% for all values of t for 

channel probabilities of error of 10~ 2 ' 5 and 10~ 3 * 0 . For p c = 

10 the recovery rate is still greater than 99% for c = 0.29 

-15 

and 0.5. For p = 10 a recovery rate of 73% of the packets 

30 fore = 0.50 may still be useful for some applications. Notice 
that at higher error rates the Breadth first approach 
substantially out-performs the Depth First approach. A penalty 
is paid for this performance at lower error rates, however, where 
the computational cost of the Breadth First approach is higher 
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than the Depth First approach* 



Finally, performance of the present invention Joint Source 
Channel Coding Strategy is compared to that of three conventional 
5 schemes: 

(4/5) convolutional codes, (one with 16 states 
and one with 8 states); and 

10 (223,255) Reed Solomon Code. 



All these schemes have approximately the same rate overhead (the 
exact number of information bits per data bit are shown in Table 





Rate in f° Tmation bits 

data bits 


-NONE- 


1.000 


e = 0.29 


0.901 


e = 0.50 


0.819 


(223, 255) RS 


0.875 


| Conv, s=8 


0.800 


§ Conv, s=16 


0.800 



Table 3: Coding Rates 



The performances of the identified schemes is plotted in Fig. 11. 
Said schemes are assumed to have the same header protection. 
Note that Fig. 11 shows that the packet loss rate for two 
30 convolutional cases noted are considerably worse over the 

simulated range. Also note that the Reed Solomon code performs 
worse for low signal to noise ratios (SNR), but when the SNR 
increases to around 5.625, the Reed Solomon marginally 
out-performs the present invention approach. The SNR's used in 
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the simulations were: 



{2.368, 4.323, 5.714, 6.790} decibels. 

5 Continuing, the amount of redundancy indicated in Table 3 

shows that the convolut i.onal codes have the highest amount 
thereof, followed by the present invention scheme with e = 0/5. 
The present invention scheme with e = 0.29 has the lowest amount 
of added redundancy of the schemes compared. It should be 
10 specifically appreciated that the present invention algorithm is 
only slightly more complex than a standard Arithmetic encoding 
scheme, with the added complexity being present primarily at the 
decoder . 

15 In the Depth First approach the complexity depends almost 

completely on the number of symbol decodings that take place 
during a packet decoding, hence the complexity is slightly more 
than the average number of symbol decodings for a given SNR . 

20 As alluded to earlier, for a Breadth First approach, two 

major factors affect the complexity. The first is the average 
number of decodings that take place during the decoding of a 
packet, which remains less than M times the number of^-symbols in 
a given packet. The averages can be seen in Fig. 11. The second 

25 factor is the sorting that takes place before an expansion at a 
branch point. The average number of branch points can be 
calculated using Equation 3. At each branch point there exists 
the possibility of needing to sort if the number of current paths 
is more than half of the maximum. The complexity of the sort in 

30 the decoding used is ' Mlog 2 (M). Thus, there is a large linear 

dependence, and an M\og(M) dependence in the decoding 
complexity. 

Present invention schemes provide substantial packet 
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recovery rates at channel rates as low as 10" ' with low coding 
overhead. Such schemes are useful in hostile communication 
environments where minimal coding overhead is advantageous. The 
approach may be especially useful for mobile and wireless 
appl icat ions . 

The present invention can be applied in communication 
systems which operate based on Binary Phase Shift Keying ( BPSK ) , 
Quadrature Phase Shift Keying (QPSK) and Trellis Coded Modulation 
(TCM) etc. 

It is noted that the terminology "variable length" refers to 
the length of code words assigned to input symbols, and the 
"joint source-channel symbol encoding" refers the use of the same 
encoding means to encode "allowed alphabetic symbols" and 
"non-alphabet symbols" for use in error correction. 

Finally, it is noted that the present invention is primarily 
useful when applied with variable length symbol coding methods. 
For example Huffman coding provides coding more probably symbols 
with shorter bit sequences. Arithmetic encoders code strings of 
symbols in a sequence of bits, and Claim language structure is 
focused to apply thereto. ^ 

Having hereby disclosed the subject matter of the present 
invention, it should be obvious that many modifications, 
substitutions, and variations of the present invention are 
possible in view of the teachings. It is therefore to be 
understood that the invention may be practiced other than as 
specifically described, and should be limited in its breadth and 
scope only by the Claims. 
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